Using a new MCQ Generation methodology (CAREGen) in the UK Electricity Distribution Industry

The Construed Antonym Realisation Exercise Generation (CAREGen) methodology for generating Multiple Choice Question (MCQ) test items is described and illustrated using two case studies from the UK Electricity Distribution Industry. The first step of the methodology is to create Controlled Specific Learning Objectives (CSLOs) that define the required assessment outcomes for the generated MCQs. The CSLOs are used to identify appropriate source documents. Statistical NLP tools are then applied to the syntactic patterns within the source documents to identify complementary and antonymic pairs of words and phrases. Construal theory is then used to define 'correct'/’incorrect’ construal of these pairs in relation to the CSLOs and a template is applied to generate the test items. The component theories of CAREGen are illustrated using the TAR Induction case study. The Cable-Jointing case study is used to illustrate application of the steps of the methodology and the current evaluation method. The conclusion explains why effective evaluation of the methodology has proved so difficult and calls for the development of a suitable evaluation measure for MCQ generation systems such as CAREGen.